"Facing Page Text" for the 
Astronaut Science Advisor Presentation 

at the 

Space Station Freedom Evolution Symposium 
August 7, 1991 


Michael M. Compton 
AI Research Branch 
NASA Ames Research Center 
(415) 604-6776 




PRECEDING PAGE BLANK NOT FILMED 


"The Astronaut Science Advisor: Ground Testing During SLS-1" 


The goal of the Astronaut Science Advisor (ASA) project is to improve the scientific return of 
experiments performed in space by providing astronaut experimenters with an "intelligent 
assistant" that encapsulates much of the domain- and experiment-related knowledge 
commanded by the PI on the ground. By using expert systems technology and the availability of 
flight-qualified personal computers, it is possible to encode the requisite knowledge and make it 
available to astronauts as they perform experiments in space. The system performs four major 
functions: diagnosis and troubleshooting of experiment apparatus, data collection, protocol 
management, and detection of interesting data. 

The experiment used for development of the system measures human adaptation to 
weightlessness in the context of the neurovestibular system. This so-called "Rotating Dome" 
experiment (which was devised by Professor Laurence Young of MIT) was flown on the recent 
Spacelab Life Sciences One (SLS-1) Mission in June, 1991. This mission was used as an 
opportunity to test some of the system's functionality. Experiment data was downlinked from the 
orbiter, and the system then captured the data and analyzed it in real time. The system kept track 
of the time being used by the experiment, recognized occurrences of interesting data, summarized 
data statistically and generated potential new protocols that could be used to optimize the course 
of the experiment. The data collected during the mission is now being used to evaluate the 
system's advice and to fine-tune the system's performance in preparation for in-flight use of the 
system on SLS-2 in 1993. 

The project was made possible by NASA grant NCC 2-570 and RTOP 506-47-11 for "Crew Station 
Design," respectively from the AI Research Branch and the Human Factors Division at NASA- 
Ames. Apple Corporation also provided generous support. 



Motivation 


Perhaps the scarcest resource for manned flight experiments - on Spacelab or on the Space 

Station Freedom - will continue to be crew time. To maximize the efficiency of the crew, and to 
make use of their abilities to work as scientist collaborators as well as equipment operators, 
normally requires more training in a wide variety of disciplines than is practical. 

In a typical laboratory setting the Principal Investigator (PI) is able to exert direct control over all 
aspects of an experiment, and his or her expertise can be brought to bear as events unfold, to 
correct problems or to follow new leads. This kind of flexibility is currently lacking during space 
experimentation, both due to time and resource constraints, and due to the physical distance from 
the PI at the time of the experiment. Communication is often not sufficient or not timely enough to 
bridge this physical gap. 

Furthermore, astronauts are trained to perform a large number of experiments in different fields, 
and cannot be expected to acquire the in-depth knowledge required to deal effectively with all 
unexpected contingencies. This problem will be exacerbated in the Space Station era, with its 
longer tours and larger number of experiments. 



ASA Overview 


The primary objective of the Astronaut Science Advisor project is to improve the scientific return 
of experiments that astronauts perform in space. The approach being pursued is to use expert 
systems technology and the availability of flight-qualified personal computers to encode the 
domain- and experiment-related knowledge and make it available to the astronauts in space- 

borne laboratories. 
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Functions of the ASA 


There are four main functions that the ASA performs: 

• Data acquisition: The ASA can collect, reduce, and archive the raw experimental data 

that is generated dining experimental sessions. 

• Data quality monitoring: Ilie ASA monitors the input data streams and checks for erratic 

or "pinned" signals, and other symptoms of poor data. If poor data is detected, the system 
notifies the astronauts that there may be equipment malfunctions and allows the operator 
to invoke the troubleshooting module. 

• Detection of "interesting data": Data that is significantly different from what the 

Principal Investigator expects is noted and can be pursued in subsequent experimental 
runs. 

• Protocol management: The ASA keeps track of the time allocated to the experiment and 

dynamically generates new protocols if the experiment is running ahead of or behind 
schedule, or if there is interesting data detected. This newly proposed experiment 
protocol can then be viewed by the astronaut operator who can then decide whether to 
adopt or decline the new protocol. 


Project Team 

The ASA project team is comprised of individuals from several NASA centers and academic 
institutions. 

At NASA's Ames Research Center, ASA project leader Silvano Colombano, Michael Compton, 
and Richard Frainier work in the Artificial Intelligence Research Branch. Irving Statler works in 
the Aerospace Human Factors Research Division. 

Jurine Adolf and Tina Holden work at the Human Computer Interaction Laboratory at NASA's 
Johnson Space Center. 

Team members from the Massachusetts Institute of Technology include Professor Laurence R. 
Young, a world-renowned space scientist who has devised numerous experiments for space- 
borne laboratories. Dr. Young, whose domain- and experiment knowledge is being modeled in 
the ASA, is also Director of the Man-Vehicle Laboratory at MIT. Working with Dr. Young at 
MIT are Nicolas Groleau (a graduate student) and Peter Szolovits of the MIT Computer Science 
Department. 



System Architecture 

The ASA system is comprised of six modules: 

• The Data Acquisition Module (DAM) collects and reduces the raw data from the on-board 
experiment equipment. 

• The Data Quality Monitor (DQM) ensures that the incoming data is reliable and error-free. 

• The Protocol Manager (PM) helps keep the experiment on schedule by monitoring 

the experiment’s progress and suggesting modifications to the protocol when necessary. 

• The Interesting Data Filter (IDF) recognizes experimental data that is likely to be "interesting” 
to the PI, and helps the protocol manager suggest ways to pursue the interesting results. 

• The Diagnostic and Troubleshooting Module (DTM) helps the astronaut isolate, diagnose, and 

correct problems in the experimental apparatus. 

• The Executive Module moderates all inter-module communications, and ensures proper and 
timely allocation of system resources. 

These modules are distributed between two computers. The "Data Computer" runs the DAM and 
DQM, and is connected directly to the on-board experiment computer via an analog-to-digital 
converter. The back-end "AI Computer" runs the PM, IDF, DTM, and the Executive, and 
interfaces directly with the astronaut operator running the experiment. 



The Rotating Dome Experiment 

The experiment used for development of the system measures human adaptation to 
weightlessness in the context of the neurovestibular system. This so-called "Rotating Dome" 
experiment was devised by Professor Young and has flown on two previous Spacelab missions 
(including, of course, the recent SLS-1 Mission in June, 1991). It is also scheduled for flight 
aboard SLS-2 in May, 1993. 

The experiment apparatus consists of a hatbox-shaped dome the inside of which is covered with 
multi-colored dots. The subject inserts his or her head into the dome while the dome rotates in 
various directions and at various speeds. Shortly after the dome begins rotating, subjects cease to 
perceive that the dome is rotating and begin to sense that it is they themselves who are rotating in 
the opposite direction. This perceived self-rotation is called vection, and is recorded via a 
"joystick" which can be rotated by the subject to indicate the direction and magnitude of the 
vection. This joystick signal, along with signals from a torque sensor on a biteboard in the dome 
and electrodes on the subject's neck, constitute the raw data that is collected during the 
experiment. 

There are three experimental conditions that are tested during the in-flight dome sessions. In the 
so-called free- float condition, the only objects in contact with the subject's body are the 

biteboard inside the dome and the joystick used to indicate vection. In the neck-twist condition, 
the subject is in free-float but starts with his or her neck bent at a forty-five degree angle relative 
to their body. In the bungee condition, the subject is held down against a floorplate by a shoulder 
harness to simulate the tactile cues on the soles of the subject’s feet that occur in 1G. 



Setting up the Dome in the Spacelab Mockup 

In this picture, the MIX PI team is checking the procedure for setting up the Rotating Dome 

the Spacelab mockup at JSC. This mockup was used during the astronauts’ training. 



Testing the Dome in the Spacelab Mockup 

This picture shows one of Professor Young’s co-investigators testing the Dome apparatus after 
setup it up in the Spacelab mockup at JSC. 



Astronauts Conducting the Dome Experiment On-Orbit 

Thic r*iHhirc> ghows a frame of the snlit-screen video that was downlinked from the Spacelab 

during the Rotating Dome session on Mission Day 5. The image on the right side of the picture is 
a rear view of one of the astronauts performing the experiment (the astronaut's head can be seen 
in the dome with her shoulders and torso extending to the lower-right comer). The image on the 
left side of the picture is a closeup of the subject’s eye taken from the video camera inside the 

dome. 
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Typical Experiment Session 

A typical session for the Rotating Dome experiment lasts approximately one hour, and consists of 
setup and calibration steps, approximately six data collection runs, followed by shutdown and 
stowage steps . 

During setup, the astronauts deploy the apparatus, connect the dome and other sensors to the 
experiment computer, and test the equipment to make sure that everything is working properly. 

The actual experiment runs each consist of six thirty-second trials during which the dome rotates 
and data is collected. During each trial, the dome rotates at a certain speed (thirty, forty-five, or 
sixty degrees per second) either clockwise or counter-clockwise. Each run tests one of the 
experimental conditions (free-float, neck-twist, or bungee, as described before). Some of the 
runs, therefore, include additional steps such as attaching or adjusting the bungee harness. 

After data collection is complete, the astronauts then shut down the experiment and disassemble 
and stow the apparatus (although sometimes the dome is left deployed if it is going to be used 
later in the mission). 


T** 4s 1 I •« -a <"» Jx.f'%. s*. «a « *"»» jCa « 3 •**■& s^/T* £*> <<"». £ s%. A d? A ■&***%. #*% S*% ✓“»> /”% -a « ^<r» <a -&*& ■£1 ■! *' a 5s*1U» 4. <o-». «■■»« ^3 ./-v-wa 4»X«k ^ £«, 1 1 *%.**■ * •e--a *>m f-% w E^&rast. 4.1% >-*. JL4 i 1 

lu liiiiDiiaic iiic UDCiuxiicSS ui me rx^jrk iu aDiTuHauiD m liigiu, euit&lu.6r me XOIiowiitg ny^uuicuLai 


scenario: 


The second Rotating Dome session is being performed approximately mid-way through a 
mission. The session, which involves data collection from two subjects, is running slightly behind 
schedule. During the first Dome session on the previous day, the first subject exhibited 
interesting data. The second subject also participated in the previous session, yet exhibited erratic 
data. 

How should the current experiment protocol be refined to maximize the scientific return from this 
Dome session? 



The "Proposed" Protocol 

Here is a screen from the Protocol Manager that shows how ASA might respond in the scenario 
just described. 

On the left part of the screen is the original protocol, showing the predefined sequence of 
subjects, runs and conditions to be carried out during this session. This display indicates that the 
first subject is currently performing a neck-twist run. 

On the right part of the screen is the new protocol being proposed by the ASA in response to the 
fact that the session is a little behind schedule and the subjects' previous performance. Note that 
the ASA suggested that the second subject’s bungee run be replaced by an additional run involving 
the first subject in the free-float condition. This recommendation is made in light of the fact that 
the first subject provided interesting data the day before, while the second subject's previous data 
had been erratic. Also, substituting a free-float run for a bungee run saves approximately two 
minutes of bungee-setup time, which will help to put the session back on schedule. 
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Diagnosis and Troubles booting Example 

Another important capability provided by ASA is the ability to help astronauts identify and correct 
malfunctions with the equipment apparatus. The following is another hypothetical example of 
how the ASA might help an astronaut fix a problem with one of the electrodes used in the 
Rotating Dome experiment. 

Suppose that while preparing a new subject for a Dome run, one of the electrodes designed to 
record neck-muscle activity malfunctions. Without the ASA, the problem would probably not be 
noticed by the astronaut operator (because the electrodes are generally only tested while setting 
up the experiment apparatus initially). As a result, the bad electrode might only be recognized by 
the Principal Investigator on the ground when the "flat" signal began to print out on the stripchart 
recorder in the SMA. (Note that if this particular session happened to take place during the 
period of time when the Spacelab was out of of communication with the ground, the problem 
might not be detected until the data is downloaded that night, well after the experiment session 
has ended!) Assuming the Pis on the ground noticed the problem and were able to isolate it, they 
would probably need to request air-to-ground communications and convey a diagnosis and 
recovery procedure up to the crew (which might take up a substantial portion of the time 
remaining for the experiment itself). 

If the ASA were being used by the crew on board, it would recognize the bad signal during the first 
experimental trial and automatically notify the operator that a signal was bad. The diagnosis 
module would then estimate how long it might take to fix the problem and then either recommend 
a diagnosis procedure (such as swapping amplifiers with the other electrode) or recommend that 
the experiment session continue without the signal. In either case, the proper advice and help 
would be available to the crew immediately, regardless of whether or not the orbiter was in 
contact with the ground. 
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Support of the SLS-1 Mission 

The ASA was used in support of the Rotating Dome experiment during three major phases of the 
Spacelab Life Sciences One mission. 

During pre-flight data collection at the Baseline Data Collection Facility at JSC, the system was 
used to help collect data at launch-minus-150 days, launch-minus-75 days, launch-minus-45 days, 
launch-minus-30 days, and launch-minus-15 days. These data, which represent how the subjects 
perform under normal conditions in 1G, served as a baseline against which to compare the data 
collected in subsequent in-flight and post-flight sessions. 

During the actual mission, the Rotating Dome experiment was carried out on Mission Day 5 and 
Mission Day 6. During these experiment sessions, the ASA system was used on the ground near 
the Science Monitoring Area (SMA) at JSC. The ASA was connected to the same stream of raw 
data that was downlinked from the Spacelab and monitored by the Pis inside the SMA. 

After the mission, the ASA was used to collect post-flight data at NASA's Dryden Flight Research 
Facility at Edwards AFB, CA. These data collection sessions, which took place on retum-plus-0 
days, return-plus 1 day, return-plus-2 days, retum-plus-4 days, return-plus 7 days, and return- 
plus-10 days, measured the astronauts' responses as they re-adapted to gravity. 
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Monitoring the Dome Experiment in the SMA 
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progress during Mission Day 5. The Pis utilize a stripchart recorder and air-to-ground 
communications (when available) to help the astronauts as they conduct the experiment on orbit. 
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Operating the ASA During the On-Orbit Dome Experiment 

In the conference room outside the SMA, the ASA system monitored, in real-time, the data 
coming down from the orbiter. The computer on the right is the so-called "AI Computer" on 
which the reasoning components of the ASA reside. The computer in the middle is the so-called 
"Data Computer" that actually collects, checks, and archives the raw data before sending the 
reduced data over to the AI Computer. The portable computer on the left was also hooked up to 
the downlinked data, and monitored the digital data stream and reflected the status of the 
experimental apparatus. 



386 


Accomplish JCJTIl 6 In. t S 

The primary accomplishment of the ASA system in its testing during SLS-1 was that it proved that 

the system works under realistic conditions. 

The data collection and quality monitoring modules correctly acquired and interpreted the data, 
and were able to keep up with the real-time data stream. 

The data analysis routines correctly interpreted the data and provided meaningful quick-look 
analyses and statistical summaries of the experiment data that were printed out and taken in to 
the Pis in the Science Monitoring Area, who generally agreed with the analyses. 

The ASA generated new protocols that included steps to pursue the "interesting" data and that 
would make optimal use of the time remaining for the experiment. These new protocols were 
also printed out and taken into the SMA, where they were used by the PI team to plan for the 
possibility of subsequent experiment sessions. 
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Lessons Learned 


There were several major lessons learned during testing of the ASA: 

• Science experiments conducted in space should permit reactivity: 

Pre-defined experimental protocols don't offer the astronauts enough flexibility to deal 
effectively with unexpected problems or opportunities. The limited availability of air-to- 
ground communication and the circuitous nature of the air-to-ground links can severely 
limit the ability of Pis on the ground to help the astronauts during experiments. 

® The ASA would have been useful to the crew in-flight: 

The crew experience various equipment and scheduling problems (such as delayed session 
starts and deferred subjects). The ASA suggested protocol refinements that would have 
minimized the effect of these occurrences on the data collection. 

• Increased emphasis on set-up would have been useful: 

Development of the ASA focussed primarily on the data collection aspects of the 
experiment. However, set-up and calibration are important parts of the protocol and 
need to be addressed more thoroughly by the ASA. 

• An in-flight system could avoid many of the limitations of a ground-based system: 

The ASA, while running on the ground, was subject to the same limitations that affected 
the Pis influence on the experiment (especially loss-of-signal events and indirect 
communication with the crew). An in-flight system would have been able to avoid much 

of these problems. 
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There are significant differences between the nature of scientific experiments that are carried out 

on board the Space Shuttle and the way that future experiments are likely to be carried out 
aboard Space Station Freedom. 

Mission Duration: On the Space Shuttle, missions are generally limited to a period of seven to 
ten days. On the other hand, astronauts aboard Space Station Freedom will conduct 
experiments that may span months or even years. 

Experiment Variety: Relatively few experiments are performed on Space Shuttle missions. On 
SLS-1, which was the first Shuttle mission dedicated to life sciences experimentation, eighteen 
primary experiments were conducted, all of which were designed to study the affect of 
spaceflight on living creatures. The number and variety of experiments that are likely to be 
performed aboard Space Station, however, will be significantly greater, and may even increase 
by an order of magnitude over Shuttle experiments. 

Experiment Protocols: Protocols that dictate how experiments are conducted aboard the Space 
Shuttle are worked out years in advanced of the mission, are very tightly scripted, and are very 
hard to refine or modify once the mission is underway. However, protocols for experiments to be 
performed on Space Station must be very flexible and adaptable to the initial results. 
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Milestones 


During 1990, the ASA team focused primarily on development, integration, and testing of the 
various software modules that comprise the ASA system. 

Much of the effort during 1991 has focused on preparation for and support of the ASA testing that 
was performed in conjunction with the SLS-1 mission. This included the pre- and post-flight 
baseline data collection sessions as well as the collection of in-flight data during the mission itself. 

For the remainder of 1991 and much of next year, the ASA team will be refining the system based 
on the experience gained during the SLS-1 mission, and preparing for the planned flight of the 
ASA on board SLS-2. This will involve rehosting the software on flight-qualified portable 
Macintosh computers, and preparation for training the SLS-2 crew on operation of the ASA. The 
project team is also investigating issues involved at generalization of the system and looking at 
other potential applications of the ASA to spacebome science. 

In 1993, the system is planned for use aboard SLS-2, and will also be used during the pre- and 
post-flight data collection sessions for that flight, which is currently scheduled for May, 1993. 



Potential Applications 

Other space scientists have expressed interest in using the ASA for their experiments. These 

potential applications would serve as a means by which to help generalize the system and would 
result in a system that could be useful to a broader segment of the space science community. 

These potential applications include: 

• The Vestibular Sled Experiment: Another of Professor Young’s experiments, the 

Vestibular Sled measures human adaptation to weightlessness from a slightly different 
perspective from the Rotating Dome. The Sled experiment is planned for flight on the 
second International Microgravity Laboratory mission (IML-2) in the 1995-96 time frame. 
® Modeling of Planetary Atmospheres: An experiment devised by Tom Scattergood and 
Chris McKay involves studying the formation of organic aerosols in Titan's atmosphere 
by simulating that environment in the so-called Gas Grain Simulation Facility. 

• Cell Growth in the Weissman Apparatus: This experiment, devised by William Weissman 

and Rose Grymes, explores how the growth of lymphocytes is affected by microgravity. 

• Biomedical Monitoring and Space Research Centrifuge: These domains, being 

investigated by Robert Mah, would involve monitoring of astronaut physiology and 
conditioning , and control of experiments that use the Space Research Centrifuge. 



Conclusions: Implications for SSF 

The experience gained during the testing of the ASA in the context of the SLS-1 mission has 
implications for how automation can help maximize the scientific return of experiments carried 
out aboard Space Station Freedom. 

"Missions" aboard Space Station will be far longer than those currently possible aboard the Space 
Shuttle. These long-duration missions will permit a wider variety of experiments, which in 
combination with the smaller crews will limit the pre-flight training and ground support that will 
be available to astronauts who must perform experiments in the Space Station environment. 

Experiments that will be carried out on board the Space Station are likely to be more reliant on 
the initial experimental results than on a pre-defined protocol. This will require that the 
astronauts be capable of interpretation of preliminary scientific data in order to determine the 
subsequent course of the experiments. 

The best way to meet these needs of crew "reactivity" and reduced reliance on help from the 
ground is to use advanced automation techniques (such as ASA) to provide the crew with on- 
board assistance. 



